Field of the Invention
The specification relates to a system for determining positions in three-dimensional space. In particular, the specification relates to a system, including a projector and camera, for determining positions in three-dimensional space by pre-calculating sine waves for pixel values.
Description of the Problem
Using a projector and a camera, it is possible to calculate the position of an object in space if the object can be seen by the camera and an image can be projected on the object by the projector. The accurate position calculation requires that the position of both the camera and projector are known and the internal geometry of the camera and projector must also be known.
For the camera in a given location and orientation, each pixel of the image sensor “sees” objects and light along a single direction that can be represented by a ray emanating from the camera. Pixels that are close to each other have rays leaving the camera in a similar direction.
A projector in a given location and orientation can shine a small square of light from each pixel. Each pixel of the projector can be represented by a single ray emanating from the projector in a specific direction.
If the geometry of the rays from the projector and camera are known and the location of the two devices are known, it is possible to use the geometric information to calculate the position where a small square of light from the projector lands on an object. The camera can capture the square of light and calculate the position of the lit part of the object. The geometric intersection of the ray of light and the ray of the pixel that captures that light is a point in three-dimensional space and the three-dimensional location of that part of the object.
The intersection of two such rays is simple to calculate and is given in many texts, including in Chapter 7 of a book by O'Rourke called “Computational Geometry in C” published in 1998 by Cambridge University Press.
Finding the location of the object using one projector pixel at a time is very slow. Alternatively an image or series of images can be projected onto the object, an image of the object captured by the camera and the image can be analyzed to determine the three-dimensional shape and position of the object. A paper by Calakli et al. entitled “One-Shot Scanning using De Bruijn Spaced Grids” includes a detailed explanation of one method of acquiring a three-dimensional shape using a special image of a De Bruin-spaced grid. The image projected can be a single image or a sequence of images. The image sequence can contain a checkerboard, a spatial pattern or a series of sinusoid patterns. The location of the projector pixel displayed by the projector is calculated by finding a pattern in the camera image or by observing a sequence of changes in the camera pixels. In the spatial pattern method, the projector projects an image containing many small unique patterns, the camera captures the image, the locations of the patterns are detected in the image and a ray-ray intersection calculation is used to recover the three-dimensional location of the object as seen by the camera.
In the spatial speckle pattern method, the projector projects a speckle pattern where the patterns of speckles each serve as a reference point from which changes in the phase of the light scattered from a rough surface can be tracked. Changes in the intensity of the speckle pattern are analyzed to determine displacements and rotations by comparing a shift between two intensity speckle patterns captured at two different times.
In the sinusoidal method which is typically called “Phase Shifting Profilometry”, the projector projects a series of images containing sinusoidal pattern on the object and the camera captures the image. Using special techniques well known to those with ordinary skill in the art, the camera can identify the angle or phase of the sinusoidal image seen by each pixel of the camera. If the sinusoidal images vary horizontally but are constant in the vertical direction, the identified angle tells the system which column of projector pixels the camera pixel is observing. In this case, the column of pixels represents a surface instead of a single ray emanating from the projector. The system can calculate the intersection of the surface represented by the column of projector pixels and the ray represented by the single camera pixel. The intersection point represents the three-dimensional point in space where the object was illuminated by the projector and captured by the camera.
In most projectors, a projected column or row of pixels does not represent a plane. However, a projected column or row of pixels represents an approximation of a plane. For example, projectors that have radial distortion due to lens design project a row of pixels that appear curved instead of flat. Because projector and camera lenses have radial distortion, errors are introduced into captured data. Some systems ignore the radial distortion when determining the intersection, which produces inaccurate results. Other systems try to correct the distortion through interpolation (also known as image warping), however, these calculations are still inaccurate, which limits the speed and accuracy of the three-dimensional calculations.